Markov Chain

A Markov chain is a sequence where item at $(t+1)$ only depends on the item at $(t)$. Formally: $P(X_{t+1} | X_t,\, X_{t-1},\, \dots,\, X_1) = P(X_{t+1} | X_t)$ This memoryless property is called Markov property.

It's used in:

• Early text generation models
• Reinforcement Learning (TODO: Markov Decision Process)
• Forward process of the diffusion models, where it adds noise to image based on the current state only.